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(57) Abstract: A system for object identiiication that enables users utilizing an imaging device to obtain information about, select, 
purchase, or perform other operation on objects. It includes an imaging device, capable of capturing one-dimensional or two-di- 
mensional images of objects. A device capable of sending the coded image through a wired/wireless channel to remote facilities is 
provded. Algorithms and software for processing and analyzing the images and for extracting from them symbolic information such 
as digits, letters, text, logos, symbols or icons are provided. It also includes algorithms and software facilitating the identification of 
the imaged objects based on the information gathered from the image and the information available in databases. Furhter, it includes 
algorithms and software for offering various information or services to the user of the imaging device based on the information 
gathered from the image and the information available in databases. 
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IMAGE-BASED OBJECT IDENTIFICATION 

L DESCRIPTION 

J A. Related Applications 
[1] The application claims priority from a co-pending U.S. Provisional Patent 

Application Serial No. 60/299,734 filed June 22, 2001, the contents of which 

are incorporated herein by reference. 

LB. Field 

[2] A visual system and method for object identification that enables users 
utilizing an imaging device to obtain information about, select, purchase, or 
perform other operation on objects. The use may be for commerce, but is not 
limited to the context of commerce. The object will be identified by the 
image or image sequence captured wdth the imaging device. 

/.C Background 

1. Introduction 
[3] Technology has brought with it numerous methods for performing 

payments and commerce-related operations in the off-line world, and for 

gaining information about items in a conmiercial or non-commercial context. 

In the prior art, there is a separation between methods for selecting objects 

1 
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about wish to gain information, methods of making purchases, and methods of 
payment. The present invention can replace the existing methods gaining 
information, for purchasing, and/or for making payments, or may combine 
two or all of these methods, all based on the visual identification of objects 
and algorithms for processing information related to such visual identification. 

2. Examples of Prior Art Product Selection Methods 
[4] The following examples are prior art methods that may be enhanced and 

improved by application of the invention. 

a) Hand held barcode scanners: 

[5] In this case, barcode scanners are attached to a computer, PDA, 

cellular phone, or some such similar user device. The user scans the desired 
product with a barcode scanner, then the product code is extracted and used to 
identify the product for performing a commerce related operation such as 
buying the product. 

b) Selection from a catalog: 

[6] A buyer marks on a catalog, or a special form, which items and 

what quantities he/she wants to order, and sends a fax of the document as an 
order form. The fax may be embedded as an attached file in an email to the 
seller of the item. 
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3 . Examples of Prior Art Payment Methods 

[7] The following examples of prior art methods may be enhanced and 

improved by application of the invention. 

a) Cellular phones and wireless PDA's; 

[8] Cellular phones and wireless PDA's can be used for performing 

payments with the proper e-wallet software for transferring the credit card 
number, authenticating the user and verifying his password and/or performing 
biometric tests. 

b) Credit cards and smart cards 

[9] Credit cards and/or smart cards are used as proof of identity for 

performing payments. 

4. What' s the problem 

[ 1 0] The main problems with conventional methods of obtaining information 

about objects, selecting products, and making payment are: 

[11] 1 . Identification of objects is not automated in many cases, and is therefore 
limited to situations in which the seller or other information provider has 
provided a specific object reference to off-line information or other means of 
obtaining information about the object. 

[12] 2. Performing the payment operation requires a cumbersome procedure, 
such as the use of a credit card, which involves manual swiping, waiting for 
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the transaction to complete, and then signing or entering a PIN code on a 
terminal. 

[13] 3. Payments using a cellular phone or PDA, can be performed by having 
the unit communicate wirelessly using IR, Bluetooth, acoustic signals or 
cellular network wireless protocols such as GSM, CDMA, etc. The checkout 
unit must then include a conununication device that increases costs for the 
retailer and requires installation. 

[14] 4. Alternatively, payments or product selection using a cellular phone or 
PDA can be performed by having the user enter into the device a phone 
number, a web address or some other access code which is marked on the 
checkout unit (and/or the product identification code) and then establish a 
wireless connection to a remote payment management unit. The disadvantage 
of this method is that the user has to enter a relatively long number/code by 
typing, speaking or writing (e.g., with a stylus), a process that is cumbersome 
and error prone. 

IL SUMMARY 

[15] To realize the advantages discussed above, the disclosed teachings 
provide: 

[16] 1 . An imaging device, capable of capturing one-dimensional or two- 
dimensional images of objects. 
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[17] 2. A device capable of sending the coded image through a wired/wireless 
channel to remote facilities. 

[18] 3. Algorithms and software for processing and analyzing the images and 
for extracting from them symbolic information such as digits, letters, text, 
logos, symbols or icons 

[19] 4. Algorithms and software facilitating the identification of the imaged 
objects based on the information gathered from the image and the information 
available in databases. 

[20] 5. Algorithms and software for offering various information or services to 
the user of the imaging device based on the information gathered from the 
image and the information available in databases. 

[21] The present invention relates generally to image based object 
identification, and more specifically to a visual method for object 
identification. The invention enables users that utilize an imaging device to 
obtain information about, select, purchase, or perform other operation on 
objects. Each object will be identified by the image or image sequence 
captured with the imaging device. 

[22] Typical activities enabled by the invention are: 
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[23] 1 . Selecting an object for inquiring more information about it, e.g., 
requesting an independent review of a book one encounters in a book-store, 
medical information about a drug, etc. 

[24] 2. Selecting an object to add it to a virtual "shopping cart", so that the user 
may go through a store, quickly adding products to such a cart and then 
ordering them on the spot or at a later time, when reviewing the total order 
made. 

[25] 3. Selecting an object for performing comparison-shopping, finding the 
various prices for the same article of commerce by various on-line and off-line 
stores. 

[26] 4. Performing the payment operation by pointing at a properly marked 
^ cash register/check out counter or label. Information read by user device 

(which may be a mobile or fixed handheld unit, a personal computer or some 
such other device) will then be used to determine the type of service or 
product to be purchased, and the payment required. 

[27] 5. Selecting an object such as a printed discount coupon or an article of 
conunerce, in order to receive a discount on purchase of the item. 

[28] 6. Selecting a reference for a physical object (e.g., a picture of the object, 
its name, its product code, or an advertisement of the product) and then 
performing any or all of the operations detailed in 1-5. For example, the user 

6 
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may see an item of food, then price, order, and pay for the food, all based on 
the visual identification of the object and the algorithms associated with the 
invention. 

[29] 7. Selecting an object to add it to a memory. For example, the details of a 
business card are added to a personal phone book. As another example, the 
user may add the picture of a home to memory, and later use that for 
comparison shopping with pictures of other homes. 

[30] 8. Selecting an object for registration, e.g., selection of a service that 
requires the transmission of the user's details like joining a member's only 
club or participating in a lottery. 

[31] 9. Selecting an object for initiation of a phone call, connecting to a web- 
site, etc. 

[32] The present invention, described here, solves all of these problems by 
enabling users utilizing an imaging device to obtain information about, select, 
purchase, or perform other operations on objects/products in the context of 
commerce or in non-conunercial contexts. The object will be identified by the 
image or image sequence captured with the imaging device. For the act of 
purchasing, a credit card number transmitting can be based on the 
identification of the wireless device owner. Thus, the need to enter long and 
conftising codes or access numbers is eliminated, and at the same time robust 
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positive identification (including image storage as proof of purchase in cases 
of dispute) is provided. 

[33] In these respects, the visual method for object identification in commerce 
according to the present invention substantially departs from the conventional 
concepts and designs of the prior art, and in so doing provides an apparatus 
primarily developed to enable users utilizing an imaging device to obtain 
information about, select, purchase, or perform other operation on objects in 
the context of conmierce or in non-commercial contexts. The object will be 
identified by the image or image sequence captured with the imaging device. 

III. BRIEF DESCRIPTION OF THE DRAWINGS 

[34] The above objectives and advantages of the disclosed teachings will 

become more apparent by describing in detail preferred embodiments thereof 

with reference to the attached drawings in which: 
[35] Figure 1 illustrates an aspect of the invention. The selection of an object 

for the purpose of inquiring more information about it. 
[36] 1.1 A person is in a specific store (which may be physical or virtual). 
[37] 1.2 The person takes an image of a specific book with a mobile phone 

(or a PDA etc.), 

[38] 1 .3 The image (or parts of it) is sent to a remote server. The image can 
include the title, the author's name or the barcode. 



8 
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[39] 1 .4 The person receives, to his mobile phone or any other personal 
device capable of receiving information, a review of the book from a web site 
or other relevant information. 

[40] Figure 2 illustrates another aspect of the invention. The selection of an 
object for the purpose of adding it to a virtual "shopping cart" - thus allowing 
the user to go through a store, quickly adding products to such a cart and then 
ordering them on the spot or later when reviewing the total order made. 

[41 ] 2. 1 A person is in a store, for example, a grocery store. 

[42] 2.2 The person sees a product of interest 

[43] 2.3 The person takes an image of the product with a PDA or a cellular 
device. The product is added to the person's shopping list 

[44] 2.4 The person repeats actions 2.2 & 2.3 and enlarges the shopping 
list. 

[45] 2.5 The order is made based on the shopping Hst. 

[46] 2.6 The payment is made based on the shopping list. 

[47] Figure 3 illustrates another aspect of the invention. The selection of an 

object from a catalog, for the purpose of making a purchase order. 
[48] 3.1 A person looks at a fashion catalog 

[49] 3.2 A product is selected and photographed directly torn the catalog 
using a cellular telephone, PDA or any other device capable of taking a 
picture. 
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[50] 3.3 The product is added to the shopping list 

[51] 3.4 The order and payment is made based on the shopping list. 

[52] Figure 4 illustrates another aspect of the invention. Performing a payment 

operation by pointing at a properly marked cash register/check out 

counter/label. 
[53] 4. 1 A person is in a parking lot. 

[54] 4.2 The person, in this case perhaps a parking lot attendant, points at a 
plate containing information about the payment action required and takes a 
picture of the plate. 

[55] 4.3 The payment is performed and the parking lot's gate is open (It 

will be appreciated, however, that this method can be totally automated, 

without any human involvement.) 
[56] Figure 5 illustrates another aspect of the invention. Uploading a coupon 

or other evidence of a discount, to be used later when purchasing a specific 

item. 

[57] 5. 1 A person reads a newspaper containing a coupon 

[58] 5 .2 The coupon is photographed using a cellular phone, PDA or any 

other device capable of taking a picture. 
[59] 5.3 The coupon is stored. 

[60] 5.4 When a product is bought the person gets a discount using the 
coupon directly firom the cellular phone, PDA or other imaging device. 
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[6 1 ] Figure 6 illustrates the flow of events for one of the above applications 

(purchasing through a catalog), and outlines the technical issues involved in 

its implementation. 
[62] 6. 1 A person presses the UCnGO button on a wireless device to 

operate a digital camera. (It will be appreciated that the invention will operate 

with any device capable of taking a digital photograph.) 
[63] 6.2 The microprocessor inside the device takes a single photo using 

default parameters for the photographic action. 
[64] 6.3 The image is analyzed and the optimal exposure time for the next 

photo is determined. A second photo is taken according to the new 

parameters. The second photograph may be taken by the user or may be taken 

automatically by the imaging device. 
[65] 6.4 The part of interest (e.g. the serial number of the requested 

product in a catalog, the digits of a barcode, a phone number for customer 

services etc.) is extracted from the entire image and is compressed. 
[66] 6.5 The image is transmitted to and received by a base station. 
[67] 6.6 The image information and the identification of the user is 

transferred from the base station to UcnGO's servers, or other server, through 

an IP net, or other digital network. 
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